


i Large Scale 
| Optimization 

State of the Art 

Edited by 


W. W. Hager 
D. W. Hearn 
and 

P. M. Pardalos 

Centerfor Applied Optimization , 
University' of Florida , Gainesville, U.S.A. 



KLUWER ACADEMIC PUBLISHERS 

DORDRECHT / BOSTON / LONDON 


A Numerical Comparison of Barrier and 
Modified Barrier Methods For Large-Scale 
Bound-Constrained Optimization* 


Stephen G. Nash. R. Polyak, and Ariela Sofer 
George Mason University, Fairfax, VA 22030 USA 


Abstract 


When a classical barrier method is applied to the solution of a nonlinear pro- 
gramming problem with inequality constraints, the Hessian matrix of the bar- 
rier function becomes increasingly ill-conditioned as the solution is approached. 
As a result, it may be desirable to consider alternative numerical algorithms. 
We compare the performance of two methods motivated by barrier functions. 
The first is a stabilized form of the classical barrier method, where a numer- 
ically stable approximation to the Newton direction is used when the barrier 
parameter is small. The second is a modified barrier method where a barrier 
function is applied to a shifted form of the problem, and the resulting barrier 
terms are scaled by estimates of the optimal Lagrange multipliers. The con- 
dition number of the Hessian matrix of the resulting modified barrier function 
remains bounded as the solution to the constrained optimization problem is ap- 
proached. Both of these techniques can be used in the context of a truncated- 
Newton method, and hence can be applied to large problems, as well as on 
parallel computers. In this paper, both techniques are applied to problems 
with bound constraints and we compare their practical behavior. 

Keywords: nonlinear programming, barrier method, modified barrier method, 
Newton’s method. truncated-Newton method, large-scale optimization. 
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1 Introduction 

We will examine the solution of nonlinear programming problems of the form 
minimize f(x) 

subject to c,(x) > 0, i = l m . U) 

Here z = (z, , . . . , x n ) r and the functions / and { c, } will be assumed to be twice 
continuously differentiable. We have in mind cases where n is large 

The methods we will consider for solving (1) will be based on classical barrier 
functions. The constrained problem is converted to a sequence of unconstrained 

problems If the logarithmic barrier function is used, then the unconstrained problems 
nave the form 

/3(x,/i) = /(z) - P In(c,(x)), 

1=1 

involving a “barrier parameter” /x > 0. If x^) denotes a minimizer of 0(x,u) then 
under appropriate conditions, it can be shown that (as „ - 0) any limit point z* of 
e sequence { x (/x) } is a solution of (1) (see Fiacco and McCormick[5]). In addition 
he associated Lagrange multiplier estimates converge to the Lagrange multipliers at 

It is well known that the Hessian matrix of the barrier function becomes increas- 
ingly ill-conditioned as n — 0 and a solution to (1) is approached. (This will be 
discussed in more detail in Section 3.) More specifically, if k constraints are binding 
at z and 0 < k < n then 6 


lmi + cond(vV(x*(/i),/x)) = +oc. 


Thus the classical barrier method “breaks down” as the method converges to the 
solution of the original constrained problem. 

We will examine two approaches that avoid this “structural” ill-conditioning (i e 
the ill-conditioning associated with the method, as distinct from the conditioning of 
the underlying optimization problem). Both approaches solve a sequence of uncon- 
strained optimization problems involving a (possibly modified) barrier function The 
hrst uses a numerically stable approximation to the Newton direction for the classical 

Tj n"?' and Sofer M)- The second uses Polyak’s modified barrier 
method [17], which incorporates an explicit representation of the Lagrange multipli- 
ers with an extension of the feasible region. Combined, these features can alleviate 
the problem of ill-conditioning, and improve the overall rate of convergence. 

In this paper, each of these unconstrained problems will be solved using a trunca- 
ted-Newton method. In this method, the Newton equations for a search direction are 
solved approximately using the conjugate-gradient method. Why choose a truncated- 
Newton method? It is a Newton-type method, that requires only first derivatives 
(although second derivatives may be utilized if desired); it has low storage costs; 


and A. Sofer 


A iVumericaJ Comparison of Barrier and Modified Barrier Methods 


i form 


i to be twice 

.ssical barrier 
inconstrained 
ned problems 


it can be adapted to solve nonconvex problems; and it vectorizes well. Thus e 
method reduces the costs of Newton’s method while maintaining rapid convergen , 
and is therefore suitable for large-scale problems. In practice the method has pro 
to be robust, effective and competitive on a wide set of unconstrained mmimiza 

Pf0 The 1 stabilized barrier method is the same as in Nash and Sofer [14] although 
it is tested here on a larger set of problems (and using a different computer). The 
modified barrier method software is new, although it was obtained by modifying the 
software for the stabilized barrier method. Because much of the software for the two 
methods is the same, we believe that this gives a clearer comparison of the proper 

° f We'Xomplre the performance of the two methods on a set of 1000-vanable 
problems with bound constraints. Preliminary computational experience with mod - 
fied barrier methods, using either a BFGS quasi-Newton method or a conjugate grad^ 
ent algorithm as the unconstrained minimization technique, is presented by Bre.tfe d 
and Shanno [3]. Their paper presents numerical results for a set of problems 
small, but have true nonlinear constraints. 


omes increas- 


iverges to the 


2 The Truncated-Newton Method 

In both the modified barrier method and the stabilized barrier method, the uncon- 
strained subproblems will be solved using a modifed version of the l ™ ncate< * -New on 
software described in Nash and Nocedal [11]. A summary of this method will be given 
here, as applied to an unconstrained problem 

minimize /(x). 

X 

The notation V/ = V/(x) is used for the gradient of / evaluated at a P oint *. 

Given some initial guess x 0 , at the j-th iteration the new estimate x of the solution 

is given by 

x = x 4* 

The search direction p most satisfy < 0 (he., it is a descent direction for / at 

‘^“tep length o > 0 is chosen to guarantee that /<*) < /(*), along with other 
conditions designed to guarantee convergence to a local mitumizer (see t Or eg. and 
Reinboldt [16]). The particular line search algorithms used are discussed belo . 

The search direction p is computed as an approximate solution of the Newton 

eqUati ° nS (V 2 f)p * - V/ (2) 

where V 2 f = V 2 fix) is the Hessian matrix of second derivatives at the current point 
x. The approximate solution is obtained by applying the conjugate-gradient method 
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to (2). This iterative method is ‘"truncated” before the exact solution is obtained. 
On parallel computers, a block conjugate gradient method could be used to solve 
(2)i resulting in a parallel barrier method (see [13]). This idea has been applied to 
bound-constrained problems for the stabilized barrier method in [10]. 

The conjugate-gradient method corresponds to minimizing the quadratic model 
Q(P) = 2 P 1 ^ 2 fP + P 7 ^ f as a function of p over a sequence of subspaces of increasing 
dimension. These are called the Krylov subspaces. 

The truncated-Newton software used here includes automatic preconditioning 
strategies designed to accelerate convergence of the conjugate-gradient method. These 
were not modified in the computational tests used in this paper, because of the spe- 
cial form of the bound constraints. For problems with more general constraints, it 
is likely that the preconditioners would have to be adjusted to take into account the 
special structure of the barrier subproblems. Techniques for doing this are discussed 
by Nash and Sofer in [15]. 



3 The Stabilized Barrier Method 

The discussion here is adapted from [14], and presents a summary of the stabilized 
barrier method. For a more complete discussion, the reference should be consulted. 

We will assume that a strictly feasible initial guess of the solution has been pro- 
vided. For problems with bound constraints, such a point can be easily found. In 
addition, we make the following standard assumptions: (a) the feasible set is compact 
and has a non-empty interior; (b) a solution z* lies in the closure of the interior of 
the feasible region; (c) z* is a regular point of the constraints (i.e., the gradients of 
the active constraints at x are linearly independent) which satisfies the second-order 
sufficiency conditions for optimality (see Fiacco and McCormick [5]). 

The logarithmic barrier method converts the problem (1) to a sequence of uncon- 
strained problems: 

m 

minimize 0{x,p) = /(*) - ft £ ln(c,(x)) , (3) 

1=1 

for a sequence of positive barrier parameters fi -*■ 0. Let x m (fi) denote an uncon- 
strained minimizer of /?(z, /i). Under quite mild conditions it can be shown that anv 
limit point z* of the sequence x*(/z) is a solution of (1). Furthermore if we define 

*•(/*) = ^/c,(z’(/i)), 

then as z (/z) — * x’, A(/z) — ♦ A*, where A" is the vector of Lagrange multipliers 
corresponding to z* (see [5]). 

The Newton direction for the barrier subproblem (3) at the point z is obtained 
by solving 

Bp = -4, 
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where b and B are the gradient and Hessian matrix respectively, of the logarithmic 
barrier function: 

Vc. 


6 = V/-.Z-T' 

«=1 c ’ 

A V 2 c, Vc.Vcf 

B = — + cl - • 


(4) 


rr c,- 


(To simplify the formulas. / is written for /(x), etc.) If we define A, = ^/c„ then (4) 
can be expressed in the form 


b = V/-X>Vc„ 


B 


= V7-E^.v 2 c, + ^E A ? Vc - Vc ^ 


( 5 ) 


1=1 


The final term in (5) reveals the ill-conditioning in the barrier subproblem. If 
a constraint is active at the solution, and its corresponding Lagrange multiplier » 
non.se, o. then the ratio Xf/p - 00 as p - 0. Thus the Hjsr.n matrix become 
progressively more ill conditioned as the solution is approached. Th.s ill-conditioning 

was noted bv Nlurrav in [8]. . 

The stabilized barrier method avoids this ill conditioning by using an approxi- 
mation to the Newton direction for the barrier function. This approximation differs 
from the Newton direction by terms of 0(/i) and so becomes more accurate as p - 0. 
The approximation is obtained by examining the range- and null-space componen 
of the search direction, defined in terms of a “working set of constraints, analogous 
to the working set used in an active-set method for constrained optimization (see. for 
example (61) The approach we propose does not require that the Hessian matrix o 
(L barrier be formed" explicr.ly. A differed approach thar avoids the ,11 copdrlionmg 
but rhar requires explicit matrix factorizations is described by VVrrght m [1SJ. 

To develop the formulas for the search direction, «e define I to be the index set of 
those constraints that contribute to the ill conditioning of the Hess, an matrix. ^ This 
set is a prediction of the set of constraints that are binding at the solution of (lb 
Let V be the matrix whose columns are the gradients of of the constraints in I. an 
assume that N has full rank. We define D = diag(A?,i 6 I), and choose Z J a 
for the null space of N T . Let N* be a pseudo-inverse for N. (For bound-constrained 
problems, the columns of .V and Z are just columns of the identity matrix.) Finally, 

define m i 

H = V 2 / - £ A, V 2 c t + -E A ?Vc,Vc, , 

1=1 ^ 

i e the “good” part of the Hessian matrix B , omitting the ill-conditioned terms. 

' Using these definitions the Newton direction can be approximated via 

D ~ Di + UPl, 
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where 


Px = -Z(Z r H Zy } Z T b, 

A = tf # (ffp,+6), (6 1 

P2 = — (iV # ) T Z?" 1 A. 

These formulas correspond to an 0( P ) approximation to the Newton direction. (A 
related stabilized formula for the search direction was derived by Murray in [8].) 

The formulas (6) only require ( Z T HZ)~ l . In our algorithm this is implemented 
by applying the conjugate-gradient method to 

Z(Z T HZ)- l Z T px = -b, 

with the iteration truncated as in the unconstrained case. The costs of finding the 
search direction in this approach are comparable to those of a naive barrier method 
that does not deal with the ill conditioning. The approximate direction obtained us- 
ing the formulas (6), together with a truncated conjugated-gradient iteration, can be 
shown to be a descent direction for the barrier function under appropriate assump- 
tions. 

A number of computational enhancements were used to improve the performance 
of the stabilized barrier method. These are discussed briefly in Section 5. 


4 The Modified Barrier Method 

We now describe the modified barrier method for the constrained problem (1). An 
extensive discussion of the theory of modified barrier methods can be found in the 
paper by Polyak [17]. 

At each major iteration of the modified barrier method the unconstrained problem 

minimize A4(x, A, p) (7) 


is solved where 

M(x, A,/i) = f(x) - + l), 

•sl 

and the solution i is used to update { A t via 

A, = + l). (8) 

The parameters { A, } are estimates of the Lagrange multipliers at the solution rV 
The function 0 is a monotone, strictly concave, and twice continuously differentiable 
function defined on the interval (0,+oo); one possible choice is ip(-) — ln(-), although 



vak , and A. Sofer 


( 6 ) 

ton direction. (A 
Turray in [8].) 

,is is implemented 


osts of finding the 
ve barrier method 
ction obtained us- 
t iteration, can be 
propriate assump- 

re the performance 
ction 5. 


1 problem (1). An 
in be found in the 

mstrained problem 

(7) 


( 8 ) 


\ Sumerical Comparison of Barrier and Modified Barrier Methods 325 

our algorithm will use a more complicated definition of xp. It is also possible to use 
the inverse function = l/(-) although this choice is not tested here. 

If, for example, 0(*) = In(-), then the feasible region for (1) is equivalent to the 

set 

| j : + l) > 0 } . 

Thus the modified barrier function is the classical Lagrangian for the problem (1) 
with the constraints expressed in this equivalent form. The use of the barrier term 

+ l) 

corresponds to perturbing the constraints so that they have the form 

*{x) > -V" 

This represents an expansion of the feasible region. Hence the implied feasible 
region” for the modified barrier subproblem varies with the barrier parameter p. 

Unlike the classical logarithmic barrier function, the modified barrier function and 
its derivatives exist at a solution x m for any positive barrier parameter p. In particular, 
if A* is the vector of Lagrange multipliers corresponding to x*, and if 0(-) = ln(-), 
then the modified barrier function has the following properties for any p > 0: 

PI. M{x 9 , = f{x m ) 

P2. =V/( i‘) - f>*Vc,(x*) = 0 

1=1 

P3. X %/i) = V 2 /(x‘) - t A:v a ci(**) + yT l £ A-Vc,(x-)Vc,(x*) r 

1=1 t=l 

When the problem is a a convex program, it follows from P2 that 

P4. x* = arg min { M{x, A*,/i) } for any p > 0. 

This means that if the optimal Lagrange multipliers A* are known, one can solve the 
constrained problem (1) using a single unconstrained optimization problem regardless 
of the value of the barrier parameter. Moreover, if the constrained optimization 
problem is nonconvex but the second-order sufficiency and strict complementarity 
conditions are satisfied at x* then there exists a p. and a p > 0 such that: 

P5. min eigV^M(x", A*,/x) > p for < p. 


t 


at the solution x*. 
lously differentiable 
•) = In(-), although 


Thus it is again possible to solve (1) using a single unconstrained optimization problem 
of the form (7) provided that the barrier parameter is sufficiently small. Of course, 
in practice only a local minimizer may be found. 
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Polyak (1 7] has shown that if the initial Lagrange multipliers are positive, and the 
barrier parameters are below some threshhold value fi, then the method converges 
Furthermore, for sufficiently small p , the successive iterates satisfy 

max { || x - x*|| . || A - A*|| } < cp ||A - A*|| . 

The constant c > 0 is independent of pt < ft. 

For a convex programming problem it is possible to prove a further result. Under 
mild conditions on the primal and dual feasible regions the modified barrier method 
converges for any fixed positive value of the barrier parameter p, provided that the 
initial vector of Lagrange multipliers is positive (see Jensen and Polvak [7]). This 
is indeed a strong result. Unlike the classical barrier method, where convergence is 
obtained by driving the barrier parameter to zero, in the modified barrier method 
convergence will occur regardless of the value of the barrier parameter 

The result (9) shows that the modified barrier method converges at a superlinear 
rate if the barrier parameter is changed from subproblem to subproblem in such a way 
that p — 0. However it is not necessary that p — 0 in order to achieve convergence; it 
is only necessary that p be reduced below the threshhold value fi. Thus the condition 
number of the Hessian matrix of the modified barrier function can remain bounded 
as the solution is approached, unlike in the classical case. 

On practical problems, it is not possible to know a priori whether t(ie initial pa- 
rameter chosen is indeed below the threshhold fi , and therefore a general-purpose 
code for solving (1) must also include some mechanism for reducing the barrier pa- 
rameter. However some caution is required. If a solution x(p) to a modified barrier 
subproblem has been found, and p is reduced to a new value fi it is possible that x(p) 
will be “infeasible” for the new subproblem: ? 

c < (*(/*)) 2 —fi- 

Suppose that the function 0 is chosen as 0(.) = ln(-). Then if fi < p and a(x) < 0 it 
is possible that 

d>(fi~ l Ci(x) + l) = ln(/i _1 c,(i) + l) 

might be undefined. This limits the flexibility of the modified barrier method (it 
limits how quickly p can be reduced) and it can greatly complicate software for this 
algorithm, particularly if the constraints are nonlinear (see also [3]). 

For this reason we have chosen to use a more elaborate definition of the function 
0, a definition that varies with the value of p. In our implementation we use a 
modification that has been suggested by Ben-Tal, Tsibulevskii and Yusefovich [21. 
Let t = cfix). If t > —/i/2 then we define 

ifi{p~ i t + l) = ln(/i -1 / -f 1). 

If < < —p/2 then we define 

0(/i~'<+ 1) =q(t) 
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where q(t) is a quadratic function for which q(-p/2), q f {-fi/ 2), and q”{-fi/ 2) match 
the corresponding values for the logarithm function at the point t = -/i/2. Since 
the quadratic function does not have a singularity at — p (or at any other point), the 
barrier parameter can be reduced at any desired rate without worrying whether the 
modified barrier function will become undefined or singular. 

Our software for the modified barrier algorithm was obtained by adapting the 
software for the stabilized barrier method. The underlying unconstrained optimiza- 
tion method is the same truncated-Newton method. More specific details (chosen as 
a result of considerable numerical testing) are discussed in Section 5. 


5 Implementation 

A number of computational enhancements were used to improve the performance of 
the stabilized barrier method. We give a brief description of these enhancements and 
discuss their effect when implemented within a modified barrier method. 


ler t lpe initial pa- 
> general-purpose 
g the barrier pa- 
modified barrier 
>ossible that i(p) 


t and c,(i) < 0 it 


rrier method (it 
software for this 

i of the function 
tation we use a 
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5.1 The Line Search 

Because the logarithmic barrier function has a singularity at the boundary of the 
feasible region, standard line search algorithms based on low-order polynomial in- 
terpolation may not be effective. For example, in implementing an inverse cubic 
interpolation line search we found that an unusually large proportion (often more 
than 50%) of the overall computational effort was spent within the line search. Re- 
placing this line search by an Armijo-type strategy reduced the fraction of time spent 
in the line search but increased the overall computational effort substantially. 

For this reason we implemented a special line search devised by Murray and Wright 
[9] specifically for the logarithmic barrier function. This line search approximates the 
barrier function along the search direction with a one-dimensional function consisting 
of a quadratic term plus a logarithmic singularity. We have found this line search to 
be effective when implemented within a classical barrier method. For example, on a 
set of problems tested in [14], the special line search led to a 27% reduction in the 
overall computational effort. 

The special line search was not as beneficial when implemented within a modified 
barrier method. This may be due to the fact that our elaborate definition of 0 no 
longer has a logarithmic singularity. The line search currently implemented in our 
software is a standard line search for unconstrained minimization based on inverse 
cubic interpolation with an acceptance test based on a Wolfe condition (the “default 7 ’ 
line search for the truncated-Newton method). 
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Customized Matrix- Vector Product 


The stabilized barrier method uses a customized matrix-vector product for the conju- 
gate-gradient iteration that isolates the terms associated with the working set X. This 
f a necessary so that rounding errors from the ill-conditioned terms do not contaminate 
the well-conditioned terms in the Hessian matrix, and hence destroy the effects of the 
stabilized approximation to the Newton direction. 

If B denotes the Hessian matrix of the barrier function then the product Bu is 

computed via the formula 


Bu 


(v 2 /)«-^E 


(V 2 c,)u 

C, 




(Vcfu)Vcf 

c? 


The terms (V a f)u and ( V 2 c,)u are computed via finite differencing: 

fr ., v/(. + H-v/(x) 

(V f)u » ~ h 1 

where h is (approximately) the square root of the machine precision. It is not safe to 
applv finite differencing directly to Bu because of the singularity of the logarithmic 
function. The final summation in the formula for Bu is computed straightforwardly 
from the formulas above. When the stabilized formulas for the search direction are 
used, the product Hu must be computed. This is done in the same way, except that 
the ill-conditioned terms are omitted from the final summation. 

The modified barrier uses a similar approach, except applied to the Hessian of the 

modified barrier function. 


6 Computational Tests 

In this section we compare the modified barrier method and the stabilized barrier 
method on a set of test problems with bound constraints. 

Many of our test problems are derived from a set of unconstrained optimization 
problems; see Table 1. For more detailed information about problems 1-52, see [11]. 
Problems 54 and 55 are from [4]. The final two problems are from release 2 of the 
Minpack-2 collection [l]. They are DPJBFG (pressure in a journal bearing) and 
DEPTFG (elastic-plastic torsion). These are the only two minimization problems in 
this collection which have bound constraints that are binding at the solution. For 
problem DPJBFG we set nx = ny = y/n, ecc = 0.1, and 6 = 10. For problem 

DEPTFG we set nx — ny — \/n, and c = 5. 

The constrained problems 1-55 are as in [14].^ In each case, we first solve the 
corresponding unconstrained problem, computing x satisfying 

||s(i)IL < 10- s (l + |/(i)|) 
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Problem 

Name 

n 1 

1 

Calculus of variations l 

100, 1000 

2 

Calculus of variations 2 

100. 1000 

3 

Calculus of variations 3 

100. 1000 

6 

Generalized Rosenbrock 

100, 1000 

8 

Penalty 1 

1000 

9 

Penalty 2 

100 

10 

Penalty 3 

1000 i 

12 

Quadratic 

1000 

28 

Extended Powell singular 

1000 

30 

Trigonometric 

100 

31 

Brown almost-linear 

100 

38 

Tridiagonal 1 

1000 

39 

Linear minimal surface 

961 

40 

Boundary-value problem 

1000 

41 

Brovden tridiagonal nonlinear 

1000 

42 

Extended ENGVLl 

1000 

43 

Ext. Freudenstein and Roth 

1000 

45 

Wrong extended Wood 

1000 

48 

Extended Rosenbrock 

1000 

49 

Extended Powell 

1000 

50 

Tridiagonal 2 

1000 

51 

Trigonometric 

1000 

52 

Penalty 1 (2nd version) 

1000 

54 

Toint 61 

1000 

55 

Toint 62 

1000 

102 

Minpack-2 (DJOURB) 

100, 1024 

105 

Minpack-2 (DTOR) 

100. 1024 


Table 1: List of test problems. 
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3 

3 

3 

3 

3 
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0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 
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using the standard initial point x 0 • Lower and upper bounds are then derived from 
;• If i is odd then 

-100 < x, < 100; 

■1 i is a multiple of 4 then 

(x), + 0.1 < x, < (x), + 10.0; 

if i is even but not a multiple of 4 then 

(x), - 10.0 < x, < (x), -0.1. 

Then a strictly feasibly initial point for the barrier method is generated. If (x Q ), < t, 
then (xo), = ii + 0.5; if (x 0 ), > u, then (x 0 ). = u, - 0.5. If (x 0 ). = li then (x 0 ). = 

/, + 10" 4 ; if (x 0 ). = then (x 0 ). = u, - 10" 4 . Then x 0 is used as the initial point for 

the barrier method. , 

The algorithms were programmed in Fortran 77 and the runs were made using 
double precision on an IBM 320H RISC workstation. The “stabilized algorithm 
uses the stabilized formula for the Newton direction when p is small; the modi- 
fied" algorithm uses the modified barrier method. The two methods incorporate the 

enhancements described in Section 5. . 

Both methods compute a search direction using a conjugate-gradient iteration 
terminated as in [12], using a rule based on the value of the quadratic model with 
tolerance 0 5. Both barrier methods were terminated when the norm of the comple- 
mentary slackness vector (scaled by 1 + |/(x)|) was less than e, = 10“ , and when the 
norm of the Lagrangian gradient (also scaled by 1 + |/(x)|) was less than e 2 - 10 . 

In addition, we required that the solution from the modified barrier method not be 
infeasible with respect to any constraint by more than = 10"®. 

We list here some details of the implementation for the stabilized barrier metho . 

l or further information, see Nash and Sofer [14]. 

• The line search was terminated using an Armijo-type test with parameter 

t? = 0.2. 

• The barrier parameter was updated using 1 

• The truncated-Newton method (for a given p) was terminated when the norm 

of the gradient (scaled as above) was less than e 3 = 10~ 3 , and when the smallest 
Lagrange multiplier estimate was greater than -e 4 , where e 4 = 10 . 

• The stabilized formula for the Newton direction was invoked whenjhe norm 
of the scaled complementary slackness vector was less than e s = 10 

We made manv test runs using the modified barrier method, and some of the more 
interesting ones are described below. However, we will only be providing detailed 
results for the best of these runs, for which the following parameter settings were 

used: 

• The line search was terminated using a Wolfe-type test with parameter rj = 
0.25. 
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Table 2: Results using (a) modified barrier method, (b) stabilized barrier 
method plus enhancements.” 

“Column "it” records the number of outer iterations, "Is” records the 
number of gradient evaluations used in the line search, “eg” records the 
number of gradient evaluations used in the inner iteration, and "total” 
records the total number of gradient evaluations (“Is” plus “eg”). 
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Table 3: Using the stabilized barrier method to solve problem 51 with 
n = 1000.“ 

a An * indicates subproblems where the 1-inverse formula for the search 
direction was used. Column “it” records the number of outer iterations, 
“Is” records the number of gradient evaluations used in the line search, 
“eg" records the number of gradient evaluations used in the inner iteration, 
and “total” records the total number of gradient evaluations (“Is” plus 
“eg”). The column “Gap” records the (scaled) duality gap, and the column 
“l|V£ir records the norm of the (scaled) Lagrangian function. 
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modified barrier method, fewer subproblems are used, each one is solved accurately, 
the barrier parameter is reduced slowly (and frequently need not be reduced at all). 
In Tables 3 and 4 these points are illustrated, with the two methods being applied to 
problem 51 with n = 1000. 

For completeness we also illustrate the performance of a “naive” barrier method 
on the same problem. The “naive” barrier method is simply a barrier method without 
the special enhancements. It uses a line search based on inverse cubic interpolation, 
it does not use extrapolation, special initialization of /*, or the 1-inverse formula, and 
it does not save the preconditioners from one subproblem to the next. The results 
for this method are shown in Table 5. 


7 Conclusions 
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We have compared the performance of a stabilized barrier method with the perfor- 
mance of a modified barrier method. Our past experience indicates that the stabilized 
barrier method is a robust and effective method for solving bound-constrained prob- 
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‘Column “it" records the number of outer iterations “l«” , , , 

number of gradient evaluations used in the line search “cv” 1 e 

number of gradient evaluations used in the inner iteration’ and “tot” * ‘5* 
the total number of gradient evaluates (“Is" plus eg") The S° 
Gap records the (scaled) duality gap, the column “llvlir™ a T" 
norm of the (scaled) Lagrang.an fusion, and 

e m nity norm of the mfeasibilit.es with respect to the bound constraints. 
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Table 5: Using a naive barrier method to solve problem 51 with 
n = 1000. a 

“Column “it” records the number of outer iterations “Is” 
records the number of gradient evaluations used in the line search 
eg records the number of gradient evaluations used in the inner 
iteration, and “total” records the total number of gradient evalua- 
ions ( Is plus “eg"). The column “Gap” records the (scaled) du- 
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lems. Our software for the stabilized barrier method is a result of much testing 
and enhancement, and represents a considerable improvement over naive barrier 
techniques. In contrast, our software for the modified barrier method is less sophis- 
ticated. Nevertheless, its performance is superior to the stabilized barrier method 
on the bound-constrained problems that we have tested. We expect that we may 
obtain even better performance with further testing and enhancement. This sug- 
gests that modified barrier methods are a promising tool for solving large nonlinear 
programming problems. 
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